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Abstract 

In-degree, PageRank, number of visits and other measures of Web page popularity signifieantly influenee the 
ranking of seareh results by modern seareh engines. The assumption is that popularity is elosely eorrelated 
with quality, a more elusive eoneept that is diffieult to measure direetly. Unfortunately, the eorrelation 
between popularity and quality is very weak for newly-ereated pages that have yet to reeeive many visits 
and/or in-links. Worse, sinee diseovery of new eontent is largely done by querying seareh engines, and 
beeause users usually foeus their attention on the top few results, newly-ereated but high-quality pages are 
effeetively “shut out,” and it ean take a very long time before they beeome popular. 

We propose a simple and elegant solution to this problem: the introduetion of a eontrolled amount of 
randomness into seareh result ranking methods. Doing so offers new pages a ehanee to prove their worth, 
although elearly using too mueh randomness will degrade result quality and annul any benefits aehieved. 
Henee there is a tradeoff between exploration to estimate the quality of new pages and exploitation of pages 
already known to be of high quality. We study this tradeoff both analytieally and via simulation, in the 
eontext of an eeonomie objeetive funetion based on aggregate result quality amortized over time. We show 
that a modest amount of randomness leads to improved seareh results. 




1 Introduction 


Search engines are becoming the predominant means of discovering and accessing content on the Web. 
Users access Web content via a combination of following hyperlinks (browsing) and typing keyword queries 
into search engines (searching). Yet as the Web overwhelms us with its size, users naturally turn to increased 
searching and reduced depth of browsing, in relative terms. In absolute terms, an estimated 625 million 
search queries are received by major search engines each day [18]. 

Ideally, search engines should present query result pages in order of some intrinsic measure of quality. 
Quality cannot be measured directly. However, various notions of popularity, such as number of in-links, 
PageRank [17], number of visits, etc., can be measured. Most Web search engines assume that popularity is 
closely correlated with quality, and rank results according to popularity. 

1.1 The Entrenchment Problem 

Unfortunately, the correlation between popularity and quality is very weak for newly-created pages that 
have few visits and/or in-links. Worse, the process by which new, high-quality pages accumulate popularity 
is actually inhibited by search engines. Since search engines dole out a limited number of clicks per unit 
time among a large number of pages, always listing highly popular pages at the top, and because users 
usually focus their attention on the top few results [11,14], newly-created but high-quality pages are “shut 
out.” This increasing “entrenchment effect” has witnessed broad commentary across political scientists, the 
popular press, and Web researchers [7-9,15,19,21] and even led to the term Googlearchy. In a recent study, 
Cho and Roy [5] show that heavy reliance on a search engine that ranks results according to popularity can 
delay widespread awareness of a high-quality page by a factor of over 60, compared with a simulated world 
without a search engine in which pages are accessed through browsing alone. 

Even if we ignore the (contentious) issue of fairness, there are well-motivated economic objectives that 
are penalized by the entrenchment effect. Assuming a notion of intrinsic page quality as perceived by users, 
a hypothetical ideal search engine would bias users toward visiting those pages of the highest quality at a 
given time, regardless of popularity. Relying on popularity as a surrogate for quality sets up a vicious cycle 
of neglect for new pages, even as entrenched pages collect an increasing fraction of user clicks. Given that 
some of these new pages will generally have higher quality than some entrenched pages, pure popularity- 
based ranking clearly fails to maximize an objective based on average quality of search results seen by 
users. 

1.2 Entrenchment Problem in Other Contexts 

The entrenchment problem may not be unique to the Web search engine context. For example, consider 
recommendation systems [13], which are widely used in e-commerce [20]. Many users decide which items 
to view based on recommendations, but these systems make recommendations based on user evaluations of 
items they view. This circularity leads to the well-known cold-start problem, and is also likely to lead to 
entrenchment. 

Indeed, Web search engines can be thought of as recommendation systems that recommend Web pages. 
The entrenchment problem is particularly acute in the case of Web search, because the sheer size of the Web 
forces large numbers of users to locate new content using search engines alone. Therefore, in this paper, we 
specifically focus on diminishing the entrenchment bias in the Web search context. 
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Figure 1: Improvement in overall quality due to rank promotion in live study. 

1.3 Our Key Idea: Rank Promotion 

We propose a very simple modification to the method of ranking search results according to popularity: 
promote a small fraction of unexplored pages up in the result list. A new page now has some chance of 
attracting clicks and attention even if the initial popularity of the page is very small. If a page has high 
quality, the rank boost gives the page a chance to prove itself. (Detailed definitions and algorithms are given 
later in the paper.) 

As an initial test for effectiveness, we conducted a real-world study, which we now describe briefly (a 
complete description is provided in Appendix We created our own small Web community consisting of 
several thousand Web pages, each containing a joke/quotation gathered from online databases. We decided 
to use “funniness” as a surrogate for quality, since users are generally willing to provide their opinion about 
how funny something is. Users had the option to rate the funniness of the jokes/quotations they visit. The 
main page of the Web site we set up consisted of an ordered list of links to individual joke/quotation pages, 
in groups of ten at a time, as is typical in search engine responses. Text at the top stated that the jokes and 
quotations were presented in descending order of funniness, as rated by users of the site. 

A total of 962 volunteers participated in our study over a period of 45 days. Users were split at random 
into two user groups: one group for which a simple form of rank promotion was used, and one for which 
rank promotion was not used. The method of rank promotion we used in this experiment is to place new 
pages immediately below rank position 20. For each user group we measured the ratio of funny votes to 
total votes during this period. Figure ^ shows the result. The ratio achieved using rank promotion was 
approximately 60% larger than that obtained using strict ranking by popularity. 

1.4 Design of Effective Rank Promotion Schemes 

In the search engine context it is probably not appropriate to insert promoted pages at a consistent rank 
position (lest users learn over time to avoid them). Hence, we propose a simple randomized rank promotion 
scheme in which promoted pages are assigned randomly-chosen rank positions. 

Still, the question remains as to how aggressively one should promote new pages. Many new pages on 
the Web are not of high quality. Therefore, the extent of rank promotion has to be limited very carefully, 
lest we negate the benefits of popularity-based ranking by displacing pages known to be of high quality too 
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often. With rank promotion there is an inherent tradeoff between exploration of new pages and exploitation 
of pages already known to be of high quality. We study how to balance these two aspects, in the context 
of an overarching objective of maximizing the average quality of search results viewed by users, amortized 
over time. In particular we seek to answer the following questions: 

• Which pages should be treated as candidates for exploration, i.e., included in the rank promotion 
process so as to receive transient rank boosts? 

• Which pages, if any, should be exploited unconditionally, i.e., protected from any rank demotion 
caused by promotion of other pages? 

• What should be the overall ratio of exploration to exploitation? 

Before we can begin to address these questions, we must model the relationship between user queries and 
search engine results. We categorize the pages on the Web into disjoint groups by topic, such that each page 
pertains to exactly one topic. Let V be the set of pages devoted to a particular topic T (e.g., “swimming” 
or “Linux”), and let U denote the set of users interested in topic T. We say that the users lA and pages V 
corresponding to topic T, taken together make up a Web community. (Users may participate in multiple 
communities.) For now we assume all users access the Web uniquely through a (single) search engine. (We 
relax this assumption later in Section |8]) We further assume a one-to-one correspondence between queries 
and topics, so that each query returns exactly the set of pages for the corresponding community. Although 
far from perfect, we believe this model preserves the essence of the dynamic process we seek to understand. 

Communities are likely to differ a great deal in terms of factors like the number of users, the number 
of pages, the rate at which users visit pages, page lifetimes, etc. These factors play a significant role in 
determining how a given rank promotion scheme influences page popularity evolution. For example, com¬ 
munities with very active users are likely to be less susceptible to the entrenchment effect than those whose 
users do not visit very many pages. Consequently, a given rank promotion scheme is bound to create quite 
different outcomes in the two types of communities. In this paper we provide an analytical method for pre¬ 
dicting the effect of deploying a particular randomized rank promotion scheme in a given community, as a 
function of the most important high-level community characteristics. 

1.5 Experimental Study 

We seek to model a very complex dynamical system involving search engines, evolving pages, and user 
actions, and trace its trajectory in time. It is worth emphasizing that even if we owned the most popular 
search engine in the world, “clean-room” experiments would be impossible. We could not even study the 
effect of different choices of a parameter, because an earlier choice would leave large-scale and indelible 
artifacts on the Web graph, visit rates, and popularity of certain pages. Therefore, analysis and simulations 
are inescapable, and practical experiments (as in Section (TaI must be conducted in a sandbox. 

Through a combination of analysis and simulation, we arrive at a particular recipe for randomized rank 
promotion that balances exploration and exploitation effectively, and yields good results across a broad 
range of community types. Robustness is desirable because, in practice, communities are not disjoint and 
therefore their characteristics cannot be measured reliably. 

1.6 Outline 

In Section |3]we present our model of Web page popularity, describe the exploration/exploitation tradeoff as 
it exists in our context, and introduce two metrics for evaluating rank promotion schemes. We then propose 
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a randomized method of rank promotion in Section |4] and supply an analytical model of page popularity 
evolution under randomized rank promotion in Section |5l In Sections EE we present extensive analytical 
and simulation results, and recommend and evaluate a robust recipe for randomized rank promotion. 

2 Related Work 

The entrenchment effect has been attracting attention for several years [7-9,15,19,21], but formal models 
for and analysis of the impact of search engines on the evolution of the Web graph [4] or on the time taken 
by new pages to become popular [5] are recent. 

A few solutions to the entrenchment problem have been proposed [3,6,22]. They rely on variations of 
PageRank: the solutions of [3,22] assign an additional weighting factor based on page age; that of [6] uses 
the derivative of PageRank to forecast future PageRank values for young pages. 

Our approach, randomized rank promotion, is quite different in spirit. The main strength of our approach 
is its simplicity—it does not rely on measurements of the age or PageRank evolution of individual Web 
pages, which are difficult to obtain and error-prone at low sample rates. (Ultimately, it may make sense to 
use our approach in conjunction with other techniques, in a complementary fashion.) 

The exploration/exploitation tradeoff that arises in our context is akin to problems studied in the field 
of reinforcemenl learning [12]. However, direcf applicafion of reinforcemenl learning algorifhms appears 
prohibifively expensive af Web scales. 

3 Model and Metrics 

In fhis secfion we infroduce fhe model of Web page popularity, adopted from [5], fhaf we use in fhe resf 
of fhis paper. (For convenience, a summary of fhe nofafion we use is provided in Table [2) Recall from 
Secfion 11.41 fhaf in our model fhe Web is cafegorized info disjoinf groups by fopic, such fhaf each page 
perfains fo exacfly one fopic. Lef V be fhe sef of pages devoted fo a parficular fopic T, and lef U denote 
fhe sef of users inferesfed in fopic T. Lef n = \V\ and u = \L{\ denote fhe number of pages and users, 
respecfively, in fhe community. 

3.1 Page Popularity 

In our model, lime is divided info discrele inlervals, and al fhe end of each inlerval fhe search engine 
measures fhe popularity of each Web page according fo in-link counf, PageRank, user fraffic, or some ofher 
indicafor of popularity among users. Usually if is only possible fo measure popularify among a minority 
of users. Indeed, for in-link counf or PageRank, only fhose users who have fhe abilify fo create links are 
counted. For mefrics based on user fraffic, typically only users who agree fo inslall a special toolbar fhaf 
monifors Web usage, as in [1], are counted. Lef Um U U denote fhe sef of monitored users, over which page 
popularify is measured, and lef m = \Um\- We assume Um consfilufes a represenlalive sample of fhe overall 
user population U. 

Lef fhe fofal number of user visifs to pages per unif fime be fixed al Vu- Further, lef v denole fhe number 
of visifs per unif time by monitored users, wifh v = Vu-^- The way Ihese visifs are dislribufed among pages 
in V is delermined largely by fhe search engine ranking melhod in use; we will come back fo fhis aspecl 
laler. For now we simply provide a definifion of fhe visif rate of a page p £ V. 
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Symbol 

Meaning 

V 

Set of Web pages in eommunity 

n 

= \V\ 

U 

Set of users in eommunity 

u 

= \U\ 

Idm 

Set of monitored users in eommunity 

m 

~ \^m\ 

Pip, t) 

Popularity among monitored users of 
page p at time t 

yu{p,t) 

Number of user visits to page p 
during unit time interval at t 

V(p, t) 

Number of visits to p by monitored users 
at t 

Vu 

Total number of user visits per unit time 

V 

Number of visits by monitored users per 
unit time 

Aip,t) 

Awareness among monitored users of 
page p at time t 

Qip) 

Intrinsie quality of page p 

1 

Expeeted page lifetime 


Table 1: Notation used in this paper. 


Definition 3.1 (Visit Rate) The visit rate of page p at time t, V (p, t), is defined as the number of times p is 
visited by any monitored user within a unit time interval at time t. 

Similarly, let Vu{p, t) denote the number of visits by any user in U (monitored and unmonitored users alike) 
within a unit time interval at time t. We require that \/t, ^u{p, t) = Vu and Vf, ^ 

Onee a user visits a page for the first time, she beeomes “aware” of that page. 

Definition 3.2 (Awareness) The awareness level of page p at time t, A{p,t), is defined as the fraction of 
monitored users who have visited p at least once by time t. 

We define fhe popularify of page p af time t, P{p, t) G [0,1], as follows: 


(1) P{p,t) = A(p,t) ■ Q(p) 

where Q{p) G [0,1] (page quality) denofes fhe exfenf fo whieh an average user would “like” page p if she 
was aware of p. 

In our model page popularity is a monofonieally nondeereasing funelion of time. Therefore if we assume 
nonzero page viewing probabilifies, for a page of infinite lifelime limj^oo P{p, t) = Qip)- 

3.2 Rank Promotion 

If pages are ranked sfriefly aeeording to eurrent popularity, it ean take a long time for the popularity of a 
new page to approaeh its quality. Artifieially promoting the rank of new pages ean potentially aeeelerate this 
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Figure 2: Exploration/exploitation tradeoff. 


process. One important objective for rank promotion is to minimize the time it takes for a new high-quality 
page to attain its eventual popularity, denoted TBP for “time to become popular.” In this paper we measure 
TBP as the time it takes for a high-quality page to attain popularity that exceeds 99% of its quality level. 

Figure|2shows popularity evolution curves for a particular page having very high quality created at time 
0 with lifetime I, both with and without rank promotion. (It has been shown [5] that popularity evolution 
curves are close to step-functions.) Time is plotted on the x-axis. The y-axis plots the number of user 
visits per time unit. Note that while the page becomes popular earlier when rank promotion is applied, 
the number of visits it receives once popular is somewhat lower than in the case without rank promotion. 
That is because systematic application of rank promotion inevitably comes at the cost of fewer visits to 
aheady-popular pages. 


3.3 Exploration/Exploitation Tradeoff and Quality-Per-Click Metric 

The two shaded regions of Figure |2l indicate the positive and negative aspects of rank promotion. The 
exploration benefit area corresponds to the increase in the number of additional visits to this particular 
high-quality page during its lifetime made possible by promoting it early on. The exploitation loss area 
corresponds to the decrease in visits due to promotion of other pages, which may mostly be of low quality 
compared to this one. Clearly there is a need to balance these two factors. The TBP metric is one-sided 
in this respect, so we introduce a second metric that takes into account both exploitation and exploitation: 
quality-per-click, or QPC for short. QPC measures the average quality of pages viewed by users, amortized 
over a long period of time. We believe that maximizing QPC is a suitable objective for designing a rank 
promotion strategy. 

We now derive a mathematical expression for QPC in our model. First, recall that the number of visits 
by any user to page p during time interval t is denoted Vu{p, t). We can express the cumulative quality of 
all pages in V viewed at time t as Ylp&v ^u{p, t) ■ Q{p)- Taking the average across time in the limit as the 
time duration tends to infinity, we obtain: 


t 


^°°ti=0 p£P 
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By normalizing, we arrive at our expression for QPC: 


'Zti=Q{T.p(^v^u{p,ti)) 

4 Randomized Rank Promotion 

We now deseribe our simple randomized rank promotion seheme (this deseription is purely eoneeptual; 
more effieient implementation teehniques exist). 

Let V denote the set of n responses to a user query. A subset of those pages, Vp CV is set aside as the 
promotion pool, whieh eontains the set of pages seleeted for rank promotion aeeording to a predetermined 
rule. (The partieular rule for seleeting Vp, as well as two additional parameters, k > 1 and r G [0,1], are 
eonfiguration options that we diseuss shortly.) Pages in Vp are sorted randomly and the result is stored in 
the ordered list Cp. The remaining pages {V — Vp) are ranked in the usual deterministie way, in deseending 
order of popularity; the result is an ordered list Cd- The two lists are merged to ereate the final result list L 
aeeording to the following proeedure: 

1. The top k — 1 elements of Cd are removed from Cd and inserted into the beginning of C while 
preserving their order. 

2. The element to insert into C at eaeh remaining position i = k,k + 1,... ,n is determined one at a 
time, in that order, by flipping a biased eoin: with probability r the next element is taken from the top 
of list Cp, otherwise it is taken from the top of Cd- If one of Cp or Cd beeomes empty, all remaining 
entries are taken from the nonempty list. At the end both of Cd and Cp will be empty, and C will 
eontain one entry for eaeh of the n pages in V. 

The eonfiguration parameters are: 

• Promotion pool {Vp): In this paper we eonsider two rules for determining whieh pages are promoted: 

(a) the uniform promotion rule, in whieh every page is ineluded in Vp with equal probability r, and 

(b) the selective promotion rule, in whieh all pages whose eurrent awareness level among monitored 
users is zero (i.e., A{p, t) = 0) are ineluded in Vp, and no others. (Other rules are of eourse possible; 
we ehose to foeus on these two in partieular beeause they roughly eorrespond to the extrema of the 
speetrum of interesting rules.) 

• Starting point {k): All pages whose natural rank is better than k are proteeted from the effeets of 
promoting other pages. A partieularly interesting value is k = 2, whieh safeguards the top result of 
any seareh query, thereby preserving the “feeling lueky” property that is of signifieant value in some 
situations. 

• Degree of randomization (r): When k is small, this parameter governs the tradeoff between empha¬ 
sizing exploration (large r) and emphasizing exploitation (small r). 

Our goal is to determine settings of the above parameters that lead to good TBP and QPC values. The re¬ 
mainder of this paper is dedieated to this task. Next we present our analytieal model of Web page popularity 
evolution, whieh we use to estimate TBP and QPC under various ranking methods. 


7 



5 Analytical Model 

Our analytical model has these features: 

• Pages have finite lifetime following an exponential distribution (Section ISTTl . The number of pages 
and the number of users are fixed in sfeady sfafe. The qualify disfribufion of pages is sfafionary. 

• The expecfed awareness, popularify, rank, and visif rafe of a page are coupled fo each ofher fhrough 
a combinafion of fhe search engine ranking function and fhe bias in user affenfion fo search resulfs 
(Sections I5^ and l5.3l . 

Given fhaf (a) modem search engines appear fo be sfrongly influenced by popularily-based measures 
while ranking resulfs, and (b) users fend fo focus fheir affenfion primarily on fhe fop-ranked resulfs [11,14], 
if is reasonable fo assume fhaf fhe expecfed visif rafe of a page is a funcfion of ifs currenf popularify (as done 
in [5]): 

(2) V{p,t) = F{P{p,t)) 

where fhe form of funcfion F{x) depends on fhe ranking mefhod in use and fhe bias in user affenfion. For 
example, if ranking is complefely random, fhen V{p, t) is independenf of P{p, t) and fhe same for all pages, 
so F{x) = V ■ (Recall fhaf v is fhe fofal number of monifored user visifs per unif time.) If ranking is done 
in such a way fhaf user Iraffic fo a page is proportional fo fhe popularify of fhaf page, F{x) = v ■ where 
(/> is a normalizafion facfor; af sfeady-sfafe, cj) = Ylpev ^)- If ranking is performed fhe aforemenfioned 
way 50% of fhe time, and performed randomly 50% of fhe time, fhen F{x) = v ■ (0.5 • | + 0.5 • ). For fhe 
randomized rank promotion we infroduced in Secfion|4]fhe sifuafion is more complex. We defer discussion 
of how fo obfain F{x) fo Section 153) 

5.1 Page Birth and Death 

The sef of pages on fhe Web is nol fixed. Likewise, we assume fhaf for a given communify based around 
topic T, fhe sef V of pages in fhe communify evolves over time due to pages being creafed and refired. To 
keep our analysis manageable we assume fhaf fhe rale of refiremenf malches fhe rale of creation, so fhaf fhe 
fofal number of pages remains fixed al n = I'Pj. We model refiremenf of pages as a Poisson process wilh 
rafe paramefer A, so fhe expecfed lifelime of a page is ( = (all pages have fhe same expecfed lifefime'). 
When a page is refired, a new page of equal qualify is creafed immedialely, so fhe disfribufion of page qualify 
values is sfafionary. When a new page is created if has initial awareness and popularify values of zero. 

5.2 Awareness Distribution 

We derive an expression for fhe disfribufion of page awareness values, which we fhen use fo obfain an 
expression for qualily-per-click (QPC). We analyze fhe sleady-slale scenario, in which fhe awareness and 
popularify dislribulions have sfabilized and remain sfeady over time. Our model may nol seem fo indicate 
sfeady-sfafe behavior, because fhe sef of pages is conslanfly in flux and fhe awareness and popularify of an 
individual page changes over time. To undersfand fhe basis for assuming sfeady-sfafe behavior, consider 
fhe sef Ct of pages creafed al lime t, and fhe sef Ct+i of pages creafed af lime t -|- 1. Since page creation 
is governed by a Poisson process fhe expected sizes of fhe Iwo sels are equal. Recall fhaf we assume fhe 

*In reality, lifetime might be a positively correlated with popularity. If so, popular pages would remain entrenched for a longer 
time than under our model, leading to even worse TBP than our model predicts. 
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Figure 3: Awareness distribution of pages of high quality under randomized and nonrandomized ranking. 


distribution of page quality values remains the same at all times. Therefore, the popularity of all pages in 
both Ct and Ct+i will increase from the starting value of 0 according to the same popularity evolution law. 
At time f +1, when the pages in Ct have evolved in popularity according to the law for the first time unit, the 
new pages in Ct+i introduced at time t+1 will replace the old popularity values of the Ct pages. A symmetric 
effect occurs with pages that are retired, resulting in steady-state behavior overall. In the steady-state, both 
popularity and awareness distributions are stationary. 

The steady-state awareness distribution is given as follows. 


Theorem 1 Among all pages in V whose quality is q, the fraction that have awareness ^ (for i = 
0,1,..., mj is: 


( 3 ) 


f{ai\q) 


_^_ TT • g) 

(A + F(0)) • (1 - at) A + F{aj • q) 


where F(x) is the function in Equation^ 


Proof: See Appendix|B] □ 

Figure |3l plots the steady-state awareness distribution for pages of highest quality, under both nonran¬ 
domized ranking and selective randomized rank promotion with k = I and r = 0.2, for our default Web 
community characteristics (see Section IhdT) . For this graph we used the procedure described in Section lOI 
to obtain the function F{x). 

Observe that if randomized rank promotion is used, in steady-state most high-quality pages have large 
awareness, whereas if standard nonrandomized ranking is used most pages have very small awareness. 
Hence, under randomized rank promotion most pages having high quality spend most of their lifetimes with 
near-100% awareness, yet with nonrandomized ranking they spend most of their lifetimes with near-zero 
awareness. Under either ranking scheme pages spend very little time in the middle of the awareness scale, 
since the rise to high awareness is nearly a step function. 

Given an awareness distribution f{a\q), it is straightforward to determine expected time-to-become- 
popular (TBP) corresponding to a given quality value (formula omitted for brevity). Expected quality-per- 
click (QPC) is expressed as follows: 


QPC = 


EpGP YT=o fiailQip)) ■ Pioi ■ Q{p)) ■ Q(p) 


'Epev Ta=o /(«ilQ(p)) ■ • Q{p)) 

where at = j-. (Recall our assumption that monitored users are a representative sample of all users.) 
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5.3 Popularity to Visit Rate Relationship 

In this section we derive the function F{x) used in Equation |2 which governs the relationship between 
P{p, t) and the expectation of V{p, t). As done in [5] we split the relationship between the popularity of a 
page and the expected number of visits into two components: (1) the relationship between popularity and 
rank position, and (2) the relationship between rank position and the number of visits. We denote these two 
relationships as the functions Fi and F 2 respectively, and write: 

F{x) = F 2 {Fi{x)) 

where the output of F\ is the rank position of a page of popularity x, and F 2 is a function from that rank to 
a visit rate. Our rationale for splitting F in this way is that, according to empirical findings reported in [11], 
the likelihood of a user visiting a page presented in a search result list depends primarily on the rank position 
at which the page appears. 

We begin with F 2 , the dependence of the expected number of user visits on the rank of a page in a result 
list. Analysis of AltaVista usage logs [5,14] reveal that the following relationship holds quite closely^: 


( 4 ) 


F2{x) = e-x-^/'^ 


where 0 is a normalization constant, which we set as: 


e 


E 


V 


n- ,-- 3/2 
1=1 ^ 


where v is the total number of monitored user visits per unit time. 

Next we turn to Fi, the dependence of rank on the popularity of a page. Note that since the awareness 
level of a particular page cannot be pinpointed precisely (it is expressed as a probability distribution), we 
express Fi (x) as the expected rank position of a page of popularity x. In doing so we compromise accuracy 
to some extent, since we will determine the expected number of visits by applying F 2 to the expected rank, 
as opposed to summing over the full distribution of rank values. (We examine the accuracy of our analysis 
in Sections l6!2l and l63l f 

Under nonrandomized ranking, the expected rank of a page of popularity x is one plus the expected 
number of pages whose popularities surpass x. By Equation 0 page p has P{p, f) > x if it has A{p, t) > 
XIQ{p). Erom Theorem ^the probability that a randomly-chosen page p satisfies fhis condition is: 


/ ( — Q{p) 

i=l+[m-3;/Q(p)J 

By linearify of expecfafion, summing over all p G T* we arrive af: 


( 5 ) 




p&V yi=l-|-Lm-a;/Q(p)J 


/ - 
m 


Q{p) 


(This is an approximafe expression because we ignore fhe effecf of fies in popularify values, and because we 
neglecf fo discounf one page of popularify x from fhe oufer summafion.) 

^User views were measured at the granularity of groups of ten results in [14], and later extrapolated to individual pages in [5]. 
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The formula for Fi under uniform randomized ranking is rather complex, so we omit it. We focus 
instead on selective randomized ranking, which is a more effective strategy, as we will demonstrate shortly. 
Under selective randomized ranking the expected rank of a page of popularity x, when x > 0, is given by: 


F[{x) 


Fi{x) if Fi{x) < k 

Fi (x) + min{ ; z} otherwise 


where Fi is as in Equation |5l and z denotes the expected number of pages with zero awareness, an estimate 
for which can be computed without difficulty under our steady-state assumption. (The case of x = 0 must 
be handled separately; we omit the details due to lack of space.) 

The above expressions for Fi(x) or F[{x) each contain a circularity, because our formula for f{a\q) 
(Equation |3ll contains F{x). It appears that a closed-form solution for F{x) is difficult to obtain. In the 
absence of a closed-form expression one option is to determine F{x) via simulation. The method we use is 
to solve for F{x) using an iterative procedure, as follows. 

We start with a simple function for F{x), say F{x) = x, as an initial guess at the solution. We then 
substitute this function into the right-hand side of the appropriate equation above to produce a new F{x) 
function in numerical form. We then convert the numerical F{x) function into symbolic form by fitting a 
curve, and repeat until convergence occurs. (Upon each iteration we adjust the curve slightly so as to fit the 
extreme points corresponding to x = 0 and x = 1 especially carefully; details omitted for brevity.) Inter¬ 
estingly, we found that using a quadratic curve in log-log space led to good convergence for all parameter 
settings we tested, so that: 


log F = a ■ (log x)^ + (3 ■ log X -h 7 

where a, (3, and 7 are determined using a curve fitting procedure. We later verified via simulation that across 
a variety of scenarios F{x) can be fit quite accurately to a quadratic curve in log-log space. 

6 Effect of Randomized Rank Promotion and Recommended Parameter 
Settings 

In this section we report our measurements of the impact of randomized rank promotion on search engine 
quality. We begin by describing the default Web community scenario we use in Section I6d1 Then we report 
the effect of randomized rank promotion on TBP and QPC in Sections 16.21 and l63l respectively. Easily, in 
Section l6!4l we investigate how to balance exploration and exploitation, and give our recommended recipe 
for randomized rank promotion. 

6.1 Default Scenario 

Eor the results we report in this paper, the default^ Web community we use is one having n = 10,000 pages. 
The remaining characteristics of our default Web community are set so as to be in proportion to observed 
characteristics of the entire Web, as follows. Eirst, we set the expected page lifetime to ( = 1.5 years (based 
on data from [16]). Our default Web community has u = 1000 users making a total of Vu = 1000 visits 
per day (based on data reported in [2], the number of Web users is roughly one-tenth the number of pages, 
and an average user queries a search engine about once per day). We assume that a search engine is able to 
monitor 10 % of its users, so m = 100 and v = 100 . 

^We supply results for other community types in Section^ 
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Figure 4: Effect of randomized rank promotion on TBP. 

As for page quality values, we had little basis for measuring the intrinsic quality distribution of pages 
on the Web. As the best available approximation, we used the power-law distribution reported for PageRank 
in [5], with the quality value of the highest-quality page set to 0.4. (We chose 0.4 based on the fraction of 
Internet users who frequent the most popular Web portal site, according to [18].) 

6.2 Effect of Randomized Rank Promotion on TBP 

Figure |4(a)| shows popularity evolution curves derived from the awareness distribution determined analyt¬ 
ically for a page of quality 0.4 under three different ranking methods: (1) nonrandomized ranking, (2) 
randomized ranking using uniform promotion with the starting point k = 1 and the degree of randomization 
r = 0.2, and (3) randomized ranking using selective promotion with k = 1 and r = 0.2. This graph shows 
that, not surprisingly, randomized rank promotion can improve TBP by a large margin. More interestingly 
it also indicates that selective rank promotion achieves substantially better TBP than uniform promotion. 
Because, for small r, there is limited opportunity to promote pages, focusing on pages with zero awareness 
turns out to be the most effective method. 

Figure |4(b)| shows TBP measurements for a page of quality 0.4 in our default Web community, for 
different values of r (fixing k = 1). As expected, increased randomization leads to lower TBP, especially if 
selective promotion is employed. 

To validate our analytical model, we created a simulator that maintains an evolving ranked list of pages 
(the ranking method used is configurable), and distributes user visits to pages according to Equation|4l Our 
simulator keeps track of awareness and popularity values of individual pages as they evolve over time, and 
creates and retires pages as dictated by our model. After a sufficient period of time has passed to reach 
steady-state behavior, we take measurements. 
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Figure 5; Quality-per-click (QPC) for default Web community as degree of randomization (r) is varied. 

These results are plotted in Figure |4(b)| side-by-side with our analytical results. We observe a close 
correspondence between our analytical model and our simulation.^ 

6.3 Effect of Randomized Rank Promotion on QPC 

We now turn to quality-per-click (QPC). Throughout this paper (except in Section[Sll we normalize all QPC 
measurements such that QPC = 1.0 corresponds to the theoretical upper bound achieved by ranking pages 
in descending order of quality. The graph in Figure plots normalized QPC as we vary the promotion 
rule and the degree of randomization r (holding k fixed at A: = 1), under our default Web community 
characteristics of Section 16.11 For a community with these characteristics, a moderate dose of randomized 
rank promotion increases QPC substantially, especially under selective promotion. 

6.4 Balancing Exploration, Exploitation, and Reality 

We have established a strong case that selective rank promotion is superior to uniform promotion. In this 
section we investigate how to set the other two randomized rank promotion parameters, k and r, so as to 
balance exploration and exploitation and achieve high QPC. For this purpose we prefer to rely on simulation, 
as opposed to analysis, for maximum accuracy. 

The graph in Figure 0plots normalized QPC as we vary both k and r, under our default scenario (Sec- 
tion lh.ll . As k grows larger, a higher r value is needed to achieve high QPC. Intuitively, as the starting point 
for rank promotion becomes lower in the ranked list (larger k), a denser concentration of promoted pages 
(larger r) is required to ensure that new high-quality pages are discovered by users. 

For search engines, we take the view that it is undesirable to include a noticeable amount of random¬ 
ization in ranking, regardless of the starting point k. Based on Figure]^ using only 10% randomization 
(r = 0.1) appears sufficient to achieve most of the benefit of rank promotion, as long as k is kept small 
(e.g.. A; = 1 or 2). Under 10% randomization, roughly one page in every group of ten query results is a 
new, untested page, as opposed to an established page. We do not believe most users are likely to notice this 
effect, given the amount of noise normally present in search engine results. 

"'Our analysis is only intended to be accurate for small values of r, which is why we only plot results for r < 0.2. From a 
practical standpoint only small values of r are of interest. 
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Figure 6: Qualitiy-per-click (QPC) for default Web community under selective randomized rank promotion, as degree 
of randomization (r) and starting point (k) are varied. 

A possible exception is for the topmost query result, which users often expect to be consistent if they 
issue the same query multiple times. Plus, for certain queries users expect to see a single, “correct,” answer 
in the top rank position (e.g., most users would expect the query “Carnegie Mellon” to return a link to the 
Carnegie Mellon University home page at position 1), and quite a bit of effort goes into ensuring that search 
engines return that result at the topmost rank position. That is why we include the k = 2 parameter setting, 
which ensures that the top-ranked search result is never perturbed. 

Recommendation: Introduce 10% randomization starting at rank position 1 or 2, and exclusively target 
zero-awareness pages for random rank promotion. 


1 Robustness Across Different Community Types 

In this section we investigate the robustness of our recommended ranking method (selective promotion rule, 
r = 0.1, k G {1, 2}) as we vary the characteristics of our testbed Web community. Our objectives are to 
demonstrate: (1) that if we consider a wide range of community types, amortized search result quality is 
never harmed by our randomized rank promotion scheme, and (2) that our method improves result quality 
substantially in most cases, compared with traditional deterministic ranking. In this section we rely on 
simulation rather than analysis to ensure maximum accuracy. 

7.1 Influence of Community Size 

Here we vary the number of pages in the community, n, while holding the ratio of users to pages fixed 
at ujn = 10%, fixing fhe fraction of monitored users as, m/u = 10%, and fixing fhe number of daily 
page visifs per user af Vu/u = v/m = 1. Figure |7(^ shows fhe resulf, wifh communify size n ploffed on 
fhe x-axis on a logarifhmic scale. The y-axis plofs normalized QPC for fhree differenf ranking mefhods: 
nonrandomized, selective randomized wifh r = 0.1 and k = 1, and selecfive randomized wifh r = 0.1 
and k = 2. Wifh nonrandomized ranking, QPC declines as communify size increases, because if becomes 
more difficulf for new high-qualify pages to overcome fhe enfrenchmenf effecf. Under randomized rank 
promofion, on fhe ofher hand, due to rank promofion QPC remains high and fairly steady across a range of 
communify sizes. 
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Figure 7: Robustness across different community types. 

7.2 Influence of Page Lifetime 

Figure |7(b)| shows QPC as we vary the expected page lifetime I while keeping all other community char¬ 
acteristics fixed. (Recall that in our model the number of pages in the community remains constant across 
time, and when a page is retired a new one of equal quality but zero awareness takes its place.) The QPC 
curve for nonrandomized ranking confirms our infuifion: when fhere is less churn in fhe sef of pages in 
fhe communify (large 1), QPC is penalized less by the entrenchment effect. More interestingly, the mar¬ 
gin of improvement in QPC over nonrandomized ranking due to introducing randomness is greater when 
pages tend to live longer. The reason is that with a low page creation rate the promotion pool can be kept 
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small. Consequently new pages benefit from larger and more frequent rank boosts, on the whole, helping 
the high-quality ones get discovered quickly. 

7.3 Influence of Visit Rate 

The influence of the aggregate user visit rate on QPC is plotted in Figure |7(c)| Visit rate is plotted on the 
x-axis on a logarithmic scale, and QPC is plotted on the y-axis. Here, we hold the number of pages fixed at 
our default value of n = 10,000 and use our default expected lifetime value of I = 1.5 years. We vary the 
total number of user visits per day Vu while holding the ratio of daily page visits to users fixed at Vu/u = I 
and, as always, fixing the fraction of monitored users as m/u = 10%. From Figure |7(c)1 we see first of all 
that, not surprisingly, popularity-based ranking fundamentally fails if very few pages are visited by users. 
Second, if the number of visits is very large (1000 visits per day to an average page), then there is no need 
for randomization in ranking (although it does not hurt much). For visit rates within an order of magnitude 
on either side of 0.1 • n = 1000, which matches the average visit rate of search engines in general when n 
is scaled to the size of the entire Web, ^ there is significant benefit to using randomized rank promotion. 

7.4 Influence of Size of User Population 

Lastly we study the affect of varying the number of users in the community u, while holding all other 
parameters fixed: n = 10,000, I = 1.5 years, Vu = 1000 visits per day, and m/u = 10%. Note that we 
keep the total number of visits per day fixed, but vary the number of users making those visits. The idea 
is to compare communities in which most page visits come from a core group of fairly active users to ones 
receiving a large number of occasional visitors. Figure |7(d)| shows the result, with the number of users u 
plotted on the x-axis on a logarithmic scale, and QPC plotted on the y-axis. All three ranking methods 
perform somewhat worse when the pool of users is large, although the performance ratios remain about 
the same. The reason for this trend is that with a larger user pool, a stray visit to a new high-quality page 
provides less traction in terms of overall awareness. 

8 Mixed Surfing and Searching 

The model we have explored thus far assumes that users make visit to pages only by querying a search 
engine. While a very large number of surf trails start from search engines and are very short, nonnegligible 
surfing may still be occurring without support from search engines. We use the following model for mixed 
surfing and searching: 

• While performing random surfing [17], users traverse a link to some neighbor with probability (1 — 
c), and jump to a random page with probability c. The constant c is known as the teleportation 
probability, typically set to 0.15 [10]. 

• While browsing the Web, users perform random surfing with probability x. With probability (1 — x) 
users query a search engine and browse among results presented in the form of a ranked list. 

We still assume that there is only one search engine that every user uses for querying. However, this 
assumption does not significantly restrict the applicability of our model. For our purposes the effect of 
multiple search engines that present the same ranked list for a query is equivalent to a single search engine 

^According to our rough estimate based on data from [2]. 
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Figure 8: Influence of the extent of random surfing. 


that presents the same ranked list and gets a user traffic equal to the sum of the user traffic of the multiple 
search engines. 

Assuming that page popularity is measured using PageRank, under our mixed browsing model the ex¬ 
pected visit rate of a page p at time t is given by: 


V(p, t) = 

+ 


(l-x) ■F{P{p,t)) 


■((( 1 - 


P{p, t) 





Figure [8] shows absolute QPC values for different values of x (based on simulation). Unlike with other 
graphs in this paper, in this graph we plot the absolute value of QPC, because the ideal QPC value varies 
with the extent of random surfing {x). Recall that x = 0 denotes pure search engine based surfing, while 
X = 1 denofes pure random surfing. Observe fhaf for all values of x, randomized rank promofion performs 
beffer fhan (or as well as) nonrandomized ranking. If is inferesfing fo observe fhaf when x is small, random 
surfing helps nonrandomized ranking, since random surfing increases fhe chances of exploring unpopular 
pages (due fo fhe feleporfafion probabilify). However, beyond a cerfain exfenf, if does nof help as much as if 
hurfs (due fo fhe explorafion/exploifafion fradeoff as was fhe case for randomized rank promofion). 


9 Summary 

The sfandard mefhod of ranking search resulfs deferminisfically according fo popularify has a significanf 
flaw: high-qualify Web pages fhaf happen fo be new are drasfically undervalued. In fhis paper we firsl 
presenfed resulfs of a real-world sfudy which demonsfrafed fhaf diminishing fhe bias againsf new pages by 
selectively and fransienfly promofing fhem in rank can improve overall resulf qualify subsfanfially. We fhen 
showed fhrough exfensive simulafion of a wide variefy of Web communify fypes fhaf promofing new pages 
by partially randomizing rank positions (using jusf 10% randomizafion) consisfenfly leads fo much higher- 
qualify search resulfs compared wifh sfricf deferminisfic ranking. From our empirical resulfs we conclude 
fhaf randomized rank promofion is a promising approach fhaf merifs furfher sfudy and evaluation. To pave 
fhe way for furfher work, we have developed new analyfical models of Web page popularify evolufion under 
deferminisfic and randomized search resulf ranking, and infroduced formal mefrics by which fo evaluafe 
ranking mefhods. 
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A Real-World Effectiveness of Rank Promotion 

In this section we describe a live experiment we conducted to study the effect of rank promotion on the 
evolution of popularity of Web pages. 
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A.l Experimental Procedure 

For this experiment we ereated our own small Web eommunity eonsisting of several thousand Web pages 
eontaining entertainment-oriented eontent, and nearly one thousand volunteer users who had no prior knowl¬ 
edge of this projeet. 

Pages: We foeused on entertainment beeause we felt it would be relatively easy to attraet a large number 
of users. The material we started with eonsisted of a large number of jokes gathered from online databases. 
We deeided to use “funniness” as a surrogate for quality, sinee users are generally willing to provide their 
opinion about how funny something is. We wanted the funniness distribution of our jokes to mimie the 
quality distribution of pages on the Web. As far as we know PageRank is the best available estimate of the 
quality distribution of Web pages, so we downsampled our initial eolleetion of jokes and quotations to mateh 
the PageRank distribution reported in [5]. To determine the funniness of our jokes for this purpose we used 
numerieal user ratings provided by the souree databases. Sinee most Web pages have very low PageRank, 
we needed a large number of nonfunny items to mateh the distribution, so we ehose to supplement jokes 
with quotations. We obtained our quotations from sites offering insightful quotations not intended to be 
humorous. Eaeh joke and quotation was eonverted into a single Web page on our site. 

Overall site: The main page of the Web site we set up eonsisted of an ordered list of links to individual 
joke/quotation pages, in groups of ten at a time, as is typieal in seareh engine responses. Text at the top 
stated that the jokes and quotations were presented in deseending order of funniness, as rated by users of 
the site. Users had the option to rate the items: we equipped eaeh joke/quotation page with three buttons, 
labeled “funny,” “neutral,” and “not funny.” To minimize the possibility of voter fraud, onee a user had rated 
an item the buttons were removed from that item, and remained absent upon all subsequent visits by the 
same user to the same page. 

Users: We advertised our site daily over a period of 45 days, and eneouraged visitors to rate whiehever 
jokes and quotations they deeided to view. Overall we had 962 partieipants. Eaeh person who visited 
the site for the first time was assigned at random into one of two user groups (we used eookies to ensure 
eonsistent group membership aeross multiple visits, assuming few people would visit our site from multiple 
eomputers): one group for whieh rank promotion was used, and one for whieh rank promotion was not used. 
Eor the latter group, items were presented in deseending order of eurrent popularity, measured as the number 
of funny votes submitted by members of the group.^ Eor the other group of users, items were also presented 
in deseending order of popularity among members of the group, exeept that all items that had not yet been 
viewed by any user were inserted in a random order starting at rank position 21 (This variant eorresponds 
to seleetive promotion with A: = 21 and r = 1.). A new random order for these zero-awareness items was 
ehosen for eaeh unique user. Users were not informed that rank promotion was being employed. 

Content rotation: Eor eaeh user group we kept the number of aeeessible joke/quotation items fixed af 1000 
fhroughouf fhe durafion of our 45-day experimenf. However, eaeh ifem had a finife lifefime of less fhan 45 
days. Eifelimes for fhe initial 1000 ifems were assigned uniformly af random from [1,30], fo simulation a 
sfeady-sfafe sifuafion in whieh eaeh ifem had a real lifefime of 30 days. When a parfieular ifem expired we 
replaeed if wifh anofher ifem of fhe same qualify, and sef ifs lifefime fo 30 days and ifs inifial popularify fo 
zero. Af all times we used fhe same joke/quofafion ifems for bofh user groups. 

*Due to the relatively small scale of our experiment there were frequent ties in popularity values. We chose to break ties based 
on age, with older pages receiving better rank positions, to simulate a less discretized situation. 
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A.2 Results 


First, to verify that the subjects of our experiment behaved similarly to users of a search engine, we measured 
the relationship between the rank of an item and the number of user visits it received. We discovered a 
power-law with an exponent remarkably close to —3/2, which is precisely the relationship between rank 
and number of visits that has been measured from usage logs of the AltaVista search engine (see Section l53l 
for details). 

We then proceeded to assess the impact of rank promotion. For this purpose we wanted to analyze a 
steady-state scenario, so we only measured the outcome of the final 15 days of our experiment (by then all 
the original items had expired and been replaced). For each user group we measured the ratio of funny votes 
to total votes during this period. Figure [2 shows the result. The ratio achieved using rank promotion was 
approximately 60% larger than that obtained using strict ranking by popularity. 

B Proof of Theorem 1 

Because we consider only the pages of quality q and we focus on steady-state behavior, we will drop q and 
t from our notation unless it causes confusion. For example, we use /(a) and V{p) instead of f{a\q) and 
V{p,t) in our proof. 

We consider a very short time interval dt during which every page is visited by at most one monitored 
user. That is, V{p)dt < 1 for every page p. Under this assumption we can interpret V{p)dt as the probability 
that the page p is visited by one monitored user during the time interval dt. 

Now consider the pages of awareness ai = —. Since these pages are visited by at most one monitored 
user during dt, their awareness will either stay at ai or increase to Oj+i. We use Vs{ai) and Vi{ai) to 
denote the probability that that their awareness remains at ai or increases from ai to Oj+i, respectively. The 
awareness of a page increases if a monitored user who was previously unaware of the page visits it. The 
probability that a monitored user visits p is U {p)dt. The probability that a random monitored user is aware 
of p is (1 — Oj). Therefore, 


Vi{ai) = V{p)dt{l - ai) = F{P{p))dt{l - ai) 

(6) = F{qai)dt{l - ai) 

Similarly, 

(7) Vsio-i) = 1 - Fi{ai) = 1 - F{qai)dt{l - ai) 

We now compute the fraction of pages whose awareness is a* after dt. We assume that before dt, /(oj) 
and /(oj-i) fraction of pages have awareness ai and Oj_i, respectively. A page will have awareness a* after 
dt if (1) its awareness is ai before dt and the awareness stays the same or (2) its awareness is aj_i before 
dt, but it increases to a*. Therefore, the fraction of pages at awareness ai after dt is potentially 

f{ai)Vs{ai) + f{ai-i)Vi{ai-i). 

However, under our Poisson model, a page disappears with probability \dt during the time interval dt. 
Therefore, only (1 — Xdt) fraction will survive and have awareness a* after dt: 

[f{ai)'Ps{ai) + f{ai-i)'Pi{ai-i)]{l - Xdt) 
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Given our steady-state assumption, the fraction of pages at a* after dt is the same as the fraction of pages at 
a* before dt. Therefore, 


(8) /(a*) = [f{ai)Vs{ai) + f{ai-i)Vi{ai-i)]{l - Xdt). 

From Equations|^0and[8l we get 

/(gp _ (1 - Xdt)F{qai_i)dt{l - a^_i) 

/(oi-i) {X +F{qai))dt{l - Qi) 

Since we assume dt is very small, we can ignore the second order terms of dt in the above equation and 
simplify it to 


.Q. fjai) ^ F{qai_i){l - aj_i) 

fiai-i) {X +F{qai)){l - Ui) 

From the multiplication of x x • • • x we get 


fjai) ^ 1 - gp A -^(ggj-i) 

/(ao) 1 - g* A + F{qaj) 


We now compute /(go)- Among the pages with awareness oq, Vsiao) fraction will stay at oq after dt. 
Also, Xdt fraction new pages will appear, and their awareness is oq (recall our assumption that new pages 
start with zero awareness). Therefore, 

(11) /(ao) = /(go)T’5(go)(l - Xdt) + Xdt 

After rearrangement and ignoring the second order terms of dt, we get 

^ F{qao) + A ^ F(0) + A 
By combining Eq nations fT()l and IT^ we get 


f{ai) 


/(gp) 


1 - gp 
1 - g* 


i 


n 


F{qaj-i) 

X + F{qaj) 


_A_ -p-r F{qaj_i) 

{X + F{0))il-ai)f}^X + F{qaj) 


21 


















